| Input (x) | Output (y) | Application |
|---|---|---|
| Home features | Price | Real Estate |
| Ad, user info | Click on an Ad? (0/1) | Online Advertising |
| Image | Object (1, …, 10) | Photo tagging |
| Image, Radar info | Position of other cars | Autonomous driving |
| Audio | Text transcript | Speech recognition |
| English | Chinese | Machine translation |
| Voice | Voice | Human computer conversation |
Question: If you have 10 filters that are \(3 \times 3 \times 3\) in one layer of a neural network, how many parameters does that layer have?
If layer \(l\) is a convolution layer:
LeCun et al., 1998. Gradient-based learning applied to document recognition
| Activation Shape | Activation Size | # Parameters | |
|---|---|---|---|
| Input | (32, 32, 1) | 1024 | 0 |
| CONV1 (f=5, s=1) | (28, 28, 6) | 6272 | \(5 \times 5 \times 6 + 6 = 156\) |
| POOL1 (f=2, s=2) | (14, 14, 6) | 1176 | 0 |
| CONV2 (f=5, s=1) | (10, 10, 16) | 1600 | \(5 \times 5 \times 16 + 16 = 416\) |
| POOL2 (f=2, s=2) | (5, 5, 16) | 1176 | 0 |
| FC3 | (120, 1) | 120 | \(400 \times 120 + 1 = 48001\) |
| FC4 | (84, 1) | 84 | \(120 \times 84 + 1 = 10081\) |
| Softmax | (10, 1) | 10 | \(84 \times 10 +1 = 841\) |
Convolution
Pooling
Fully connected